-
Notifications
You must be signed in to change notification settings - Fork 0
[WIP] Make node renderer use ndjson format for communication #536
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
[WIP] Make node renderer use ndjson format for communication #536
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login. Comment |
- Introduced `IncrementalRenderRequestManager` to handle streaming NDJSON requests, managing state and processing of incremental render requests. - Added `validateBundlesExist` utility function to check for the existence of required bundles, improving error handling for missing assets. - Refactored the incremental render endpoint to utilize the new request manager, enhancing the response lifecycle and error management. - Updated tests to cover scenarios for missing bundles and validate the new request handling logic.
- Replaced the `IncrementalRenderRequestManager` with `handleIncrementalRenderStream` to manage streaming NDJSON requests more efficiently. - Enhanced error handling and validation during the rendering process. - Updated the `run` function to utilize the new stream handler, improving the response lifecycle and overall performance. - Removed the deprecated `IncrementalRenderRequestManager` class to streamline the codebase.
- Introduced improved error handling for malformed JSON chunks during the incremental rendering process. - Added logging and reporting for errors in subsequent chunks while allowing processing to continue. - Updated tests to verify behavior for malformed JSON in both initial and update chunks, ensuring robust error management.
…inability - Introduced helper functions to reduce redundancy in test setup, including `getServerAddress`, `createHttpRequest`, and `createInitialObject`. - Streamlined the handling of HTTP requests and responses in tests, enhancing clarity and organization. - Updated tests to utilize new helper functions, ensuring consistent structure and easier future modifications.
- Replaced inline wait functions with a new `waitFor` utility to improve test reliability and readability. - Updated tests to utilize `waitFor` for asynchronous expectations, ensuring proper handling of processing times. - Simplified the test structure by removing redundant wait logic, enhancing maintainability.
…processing - Introduced `createBasicTestSetup` and `createStreamingTestSetup` helper functions to streamline test initialization and improve readability. - Added `sendChunksAndWaitForProcessing` to handle chunk sending and processing verification, reducing redundancy in test logic. - Updated existing tests to utilize these new helpers, enhancing maintainability and clarity in the test structure.
- Added detailed error reporting in the `waitFor` function to include the last encountered error message when a timeout occurs. - Refactored the `createStreamingResponsePromise` function to improve clarity and maintainability by renaming variables and returning received chunks alongside the promise. - Updated tests to utilize the new structure, ensuring robust handling of streaming responses and error scenarios.
- Introduced `validateAndGetBundlePaths` and `buildVMsForBundles` functions to streamline bundle validation and VM building processes. - Updated `handleIncrementalRenderRequest` and `handleRenderRequest` to utilize the new validation and VM building functions, improving code clarity and maintainability. - Enhanced error handling for rendering execution and added support for incremental updates using an EventEmitter. - Created a new `sharedRenderUtils` module to encapsulate shared rendering logic, promoting code reuse and organization.
…ation" This reverts commit d3036fe.
…ment - Removed unnecessary bundle validation checks from the incremental render request flow. - Enhanced the `handleIncrementalRenderRequest` function to directly call `handleRenderRequest`, streamlining the rendering process. - Updated the `IncrementalRenderInitialRequest` type to support a more flexible structure for dependency timestamps. - Improved error handling to capture unexpected errors during the rendering process, ensuring robust responses. - Added cleanup logic in tests to restore mocks after each test case.
- Removed individual protocol version and authentication checks from the request handling flow. - Introduced a new `performRequestPrechecks` function to streamline the validation process for incoming requests. - Updated the `authenticate` and `checkProtocolVersion` functions to accept request bodies directly, enhancing modularity. - Improved error handling by ensuring consistent response structures across precheck validations.
- Updated the `/upload-assets` endpoint to differentiate between assets and bundles, allowing for more flexible uploads. - Introduced logic to extract bundles prefixed with 'bundle_' and handle them separately. - Integrated the `handleNewBundlesProvided` function to manage the processing of new bundles. - Added comprehensive tests to verify the correct handling of uploads with various combinations of assets and bundles, including edge cases for empty requests and duplicate bundle hashes.
- Added tests to verify directory structure and file presence for uploaded bundles and assets. - Implemented checks for scenarios with empty requests and duplicate bundle hashes, ensuring correct behavior without overwriting existing files. - Improved coverage of the `/upload-assets` endpoint to handle various edge cases effectively.
- Implemented a new test case for the `/upload-assets` endpoint to verify that bundles are correctly placed in their own hash directories rather than the targetBundles directory. - Ensured that the test checks for the existence of the bundle in the appropriate directory and confirms that the target bundle directory remains empty, enhancing coverage for asset upload scenarios.
- Implemented a suite of tests for the `/bundles/:bundleTimestamp/incremental-render/:renderRequestDigest` endpoint to verify successful rendering under various conditions, including pre-uploaded bundles and assets. - Added scenarios to test failure cases, such as missing bundles, incorrect passwords, and invalid JSON payloads. - Enhanced coverage for handling multiple dependency bundles and processing NDJSON chunks, ensuring robust error management and response validation.
- Simplified test structure by introducing helper functions to reduce code duplication for creating worker apps and uploading bundles. - Improved test cases for the `/bundles/:bundleTimestamp/incremental-render/:renderRequestDigest` endpoint, ensuring robust validation of successful renders and error handling for various scenarios. - Added tests for handling invalid JSON and missing required fields, enhancing coverage for edge cases in the rendering process. - Updated tests to ensure proper handling of multiple dependency bundles and improved response validation for different payload conditions.
- Replaced the `runInVM` function with a new `ExecutionContext` class to manage VM contexts more effectively. - Updated the `handleRenderRequest` function to utilize the new `ExecutionContext`, improving the handling of rendering requests. - Enhanced error management by introducing `VMContextNotFoundError` for better clarity when VM contexts are missing. - Refactored tests to align with the new execution context structure, ensuring consistent behavior across rendering scenarios.
…andling - Updated the parameters for the `runOnOtherBundle` function to ensure correct execution order. - Introduced a reference to `globalThis.runOnOtherBundle` in the server rendering code for better accessibility. - Enhanced the test fixture to align with the changes in the global context, ensuring consistent behavior across rendering requests.
- Introduced `IncrementalRenderSink` type to manage streaming updates more effectively. - Updated `handleIncrementalRenderRequest` to return an optional sink and handle execution context errors gracefully. - Refactored the `run` function to utilize the new sink for processing updates, enhancing error logging for unexpected chunks. - Simplified test setup by removing unused sink methods, ensuring tests focus on relevant functionality.
- Updated the `setResponse` call in the `run` function to correctly use `result.response`. - Expanded the incremental render tests to cover new scenarios, including basic updates, multi-bundle interactions, and error handling for malformed update chunks. - Introduced new helper functions in test fixtures to streamline the creation of async values and streams, enhancing the robustness of the tests. - Improved the secondary bundle's functionality to support async value resolution and streaming, ensuring consistent behavior across bundles.
No description provided.